chore(ai): Track reason for missing cost in ai cost calculation#5611
chore(ai): Track reason for missing cost in ai cost calculation#5611
Conversation
shellmayr
changed the title
| - Add `gen_ai.cost_calculation.result` metric to track AI cost calculation outcomes by integration and platform. ([#5560](https://github.com/getsentry/relay/pull/5560)) | ||
| - Normalizes and validates trace metric names. ([#5589](https://github.com/getsentry/relay/pull/5589)) | ||
| - Add manual category to cost calculation metric origin tag ([#5603](https://github.com/getsentry/relay/pull/5603)) | ||
| - Differentiate between reasons for missing cost calculation([#5611](https://github.com/getsentry/relay/pull/5611)) |
There was a problem hiding this comment.
Feel free to skip the changelog for changes which aren't "outside" visible
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Dav1dde
left a comment
Dav1dde
left a comment
There was a problem hiding this comment.
Can't tell if cursor is correct 
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| result = "calculation_no_model_id_available", | ||
| integration = map_origin_to_integration(origin), | ||
| platform = platform_tag(platform), | ||
| ); |
There was a problem hiding this comment.
Missing metric when model costs config is unavailable
Medium Severity
When model_costs is None in enrich_ai_span_data, no cost calculation metric is emitted. The code silently skips the extract_ai_data call, meaning no tracking occurs for this failure case. This is inconsistent with the EAP path in eap/ai.rs where the "calculation_no_model_cost_available" metric is correctly emitted when model_costs is None. This defeats the PR's purpose of differentiating between reasons for missing cost calculation.
| platform = platform_tag, | ||
| ); | ||
| return; | ||
| }; |
There was a problem hiding this comment.
EAP path missing distinct "no model ID" metric
Low Severity
The EAP path emits "calculation_no_model_cost_available" when no model ID attribute exists (GEN_AI_REQUEST_MODEL/GEN_AI_RESPONSE_MODEL), but the span path emits "calculation_no_model_id_available" for the same scenario. The PR introduces "calculation_no_model_id_available" as a distinct metric (documented in statsd.rs and used in normalize/span/ai.rs), but the EAP path doesn't distinguish this case. This inconsistency makes telemetry data harder to interpret since the same condition produces different metric values.
2 participants
Closes TET-1892